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I.  INTRODUCTION 


A.  BACKGROUND 

'Vlhe  Tri-Service  Medical  Information  System  (TRIMIS)  Program 
Office  (TPO)  has  installed  computerized  clinical  laboratory  systems 
[Tri-Service  Laboratory  (TRILAB)  systems]  at  Oakland  Naval  Regional 
Medical  Center  (Oakland  NRMC) ,  Wright  Patterson  Medical  Center  and 
Regional  Hospital  (Wright  Patterson),  and  Dwight  D.  Eisenhower  Army 
Medical  Center  (Eisenhower) .  The  costs  and  benefits  of  the  system 
were  evaluated  at  Oakland  NRMC.  Evaluations  at  the  other  two  sites 
were  conducted  on  a  smaller  scale  and  consisted  primarily  of  surveys 
of  attitudes  of  the  users  of  clinical  laboratory  services  and  of 
laboratory  personnel.  Information  from  evaluations  at  all  three  sites 
will  be  used  in  decision-making  regarding  the  future  use  of  automation 
in  clinical  laboratories  in  other  medical  treatment  facilities.  - 

This  report,  which  is  Volume  III  of  a  four-volume  report,  pre¬ 
sents  summary  results  of  the  baseline  (Period  X  or  pre-implementation 
survey)  and  detailed  results  of  the  post-implementation  (Period  Y) 
survey  of  attitudes  of  users  and  providers  of  clinical  laboratory 
services  at  Wright  Patterson  Medical  Center  and  Regional  Hospital,  and 
analyzes  changes  in  staff  perceptions  of  clinical  laboratory  services. 
The  baseline  surveys  were  administered  in  the  winter  of  1981.  Imple¬ 
mentation  of  the  TRILAB  system  began  in  June  1982.  The  post¬ 
implementation  surveys  were  carried  out  in  the  fall  of  1982.  The 
survey  questionnaires  addressed  issues  relating  to  clinical  labora¬ 
tory,  laboratory  efficiency,  frequency  of  events  occurring  in  the 
laboratory,  expected  turnaround  times  for  different  laboratory  tests, 
and  efficacy  of  information  storage  and  retrieval.  (Detailed  baseline 
survey  results  are  presented  in  Volume  IV  and  Appendix  C,  Volume  VI  of 
the  Baseline  Evaluation  Report.) ^ 

The  remainder  of  this  chapter  discusses  laboratory  operations, 
and  compares  the  TRILAB  system  with  the  Air  Force  Clinical  Laboratory 
Automated  System  (AFCLAS)  which  it  replaced.  Chapter  II  presents  the 
approach  taken  in  the  survey.  Chapter  III  presents  the  results  of  the 
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surveys  and  Chapter  IV  the  results  of  interviews  that  were  conducted 
with  providers  and  clinical  laboratory  personnel.  Appendix  C  in 
Volume  VI  contains  the  post-implementation  survey  questionnaires,  and 
the  number  of  respondents  to  each  question.  Also  included  in  the 
appendix  are  detailed  results  of  the  post-implementation  survey. 

B.  THE  SETTING 

Wright  Patterson  Medical  Center  and  Regional  Hospital  serves 
about  400,000  outpatient  visits  and  104,000  inpatient  days  (8,335 
admissions)  per  year.  The  annual  volume  of  laboratory  tests  performed 
was  approximately  1,622,000  (CY  1982),  equivalent  to  4.8  million  CAP 
units,  including  inpatient  STAT  and  routine,  outpatient  STAT  and 
routine,  and  emergency  room  STAT  tests.  The  laboratory  serves  all 
wards  and  clinics  at  the  hospital  (in-house  users) ,  in  addition  to 
several  outlying  clinics  and  hospitals. 

At  the  time  of  the  post-implementation  survey  the  laboratory  had 
a  staff  of  42.  The  laboratory  was  organized  into  six  sections: 
Automated  Chemistry,  Chemistry  Processing,  Hematology,  Microbiology, 
STAT  Laboratory  and  Urinalysis.  This  organization  had  not  changed 
from  the  baseline  period. 

C.  DESCRIPTION  OF  AFCLAS 

Prior  to  implementation  of  the  TRILAB  system,  the  clinical 
laboratory  at  Wright  Patterson  had  a  Honeywell  Air  Force  Clinical 
Laboratory  Automated  System  (AFCLAS).  Test  results  could  be  accessed 
by  CRT  terminal  in  the  laboratory,  and  by  teletypewriter  terminals  on 
a  few  (four  or  five)  wards  and  clinics.  The  Hycel  17,  a  large-volume 
chemistry  analyzer,  was  interfaced  with  AFCLAS,  and  other  results  were 
entered  into  the  system  via  terminals.  AFCLAS  was  capable  of 
highlighting  abnormal  results  and  producing  cumulative  reports  on 
patient  tests;  quality  control  and  other  laboratory  management 
statistics  were  also  retrievable.  Figure  1  illustrates  the  flow  of 
information  in  the  baseline  laboratory  process. 

In  the  baseline  system,  results  were  reported  to  providers  in  two 

ways: 
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FIGURE  1  BASELINE  WORK  FLOW  AT  WRIGHT  PATTERSON  MEDICAL  CENTER  AND  REGIONAL  HOSPITAL 
CLINICAL  LABORATORY  (HONEYWELL  AFC  LAS  SYSTEM) 
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1.  Providers  on  those  four  or  five  wards  and  clinics  with 
teletypewriter  terminals  could  obtain  results  via 
interrogation; 

2.  Printed  daily  result  reports  were  sent  to  each  ward  and 
clinic. 

The  volume  of  telephone  calls  to  the  laboratory  for  inquiry  regarding 
test  results  or  status  was  considered  high,  because  most  wards  and 
clinics  did  not  have  terminals  and,  of  those  that  did,  many  providers 
called  the  laboratory,  rather  than  attempt  to  interrogate  using  the 
teletypewriter  terminal. 

D.  DESCRIPTION  OF  TRILAB  SYSTEM  AT  WRIGHT  PATTERSON 

The  TRILAB  system  included  a  central  processing  unit,  and 
peripheral  printers  and  CRT  terminals  located  in  the  departments,  as 
summarized  in  Table  1.  The  system  included  a  total  of  25  terminals 
and  8  printers:  11  of  the  terminals  were  located  in  hospital  patient 
care  areas,  and  1  printer  and  3  terminals  were  located  in  outlying 
clinics. 

With  the  TRILAB  system,  laboratory  test  requests  were  generated 
on  wards  and  clinics  by  use  of  a  single  special  precoded  form, 
developed  internally.  Chemistry  and  Hematology  test  requests  were 
centrally  accessioned  into  the  laboratory  computer  at  the  reception 
desk;  Microbiology  requests  were  accessioned  at  the  section.  The 
precoded  form  allowed  the  receptionist  to  review  and  accession  the 
test  request  information  efficiently  into  the  computer.  The  computer 
automatically  generated  three  labels  for  specimens.  Specimens  were 
then  sent  to  the  appropriate  laboratory  station  according  to  the  type 
of  test  requested.  Figure  2  shows  the  work  flow  in  the  post¬ 
implementation  period. 

Tests  were  performed  in  the  following  order  of  priority: 
emergency  room  STAT,  inpatient  STAT,  outpatient  STAT,  inpatient 
routine,  and  outpatient  routine. 

The  Hycel  was  still  the  only  laboratory  instrument  that  had  been 
interfaced  with  the  computer.  Other  test  results  were  entered 
manually  through  terminals.  Results  were  verified  by  laboratory 
supervisors  before  release.  As  STAT  test  results  were  verified  and 
made  available,  the  terminal  at  the  app'  opriate  ward/clinic  "beeped." 
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TABLE  1 


SUMMARY  OF  TRILAB  PERIPHERAL  LOCATIONS 
WRIGHT  PATTERSON  MEDICAL  CENTER  AND  REGIONAL  HOSPITAL 


Medical  Svstems 


Laboratory 


Reception  Desk 

Automated  Chemistry 

Chemistry  Processing 

Hematology 

Microbiology 

STAT  Laboratory 

Urinalysis 


Outpatient  Clinics 


Family  Practice,  Surgery /Urology ,  ER, 
OB/GYN,  Internal  Medicine 


Inpatient  Wards 


Internal  Medicine,  North  Medical  Ward, 
North  Medical/Surgical  Ward,  Intensive 
Care  Ward,  Pediatrics,  Surgical, 
Psychiatry,  OB/GYN,  Orthopedics 


Outlying  Clinics 


Employee  Health,  Pediatric/Lab,  OB/GYN, 
Occupational  Health 
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Printers 


1 

0 

1 

1 

1 

0 
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CRTs 


2 

1 

1 

1 
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FIGURE  2  POST-IMPLEMENTATION  WORK  FLOW  AT  WRIGHT  PATTERSON  MEDICAL  CENTER  AND  REGIONAL  HOSPITAL 
CLINICAL  LABORATORY  (TRILAB  SYSTEM) 
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Routine  results  were  also  available  via  inquiry.  The  major  reports 
produced  by  the  system  were: 

•  A  daily  report  by  patient,  summarizing  the  results 
of  tests  carried  out  during  the  previous  24  hours. 

These  reports  were  printed  at  night  and  were  avail¬ 
able  after  midnight. 

•  An  inpatient  cumulative  report,  printed  on  discharge 
for  inclusion  in  the  patient  chart.  Interim  printed 
cumulative  reports  were  also  available  on  request. 

A  description  of  management  reports  that  the  system  also  produced  is 
available  in  Volume  II. 

E.  DIFFERENCES  IN  WORK  FLOW  BETWEEN  BASELINE  AND  POST-IMPLEMENTATION 
PERIODS 

The  major  differences  between  the  baseline  (AFCLAS)  and 
post-implementation  (TRILAB)  systems  were  as  follows: 

•  Test  Requests:  In  the  post-implementation  period  tests 
were  ordered  by  use  of  a  single  precoded  form.  In  the 
baseline,  tests  were  ordered  on  various  forms. 

•  Test  Results:  The  TRILAB  system  allowed  for  full 
inquiry  capability  through  use  of  terminals  on  wards 
and  clinics.  For  STAT  results,  the  terminal  of  the 
appropriate  location  "beeped"  as  they  became  available, 
eliminating  the  need  to  call  the  laboratory;  this 
capability  (notification  of  STAT  test  availability) 
was  not  available  with  AFCLAS. 

•  Flexibility:  Overall,  TRILAB  was  considered  a  much  more 
flexible  system  than  the  previous  system.  Information 
storage  and  retrieval  was  considered  more  efficient; 

in  addition  to  having  more  terminals,  results  were  more 
easily  accessible  to  both  laboratory  staff  and  users 
via  the  CRTs  than  the  teletypewriter  terminals.  In 
addition,  computer  program  changes  could  be  more  easily 
made  with  TRILAB  (via  telephone  input  from  the  vender 
to  the  site)  than  with  the  AFCLAS  system. 


/L  Arthur  D.  Little,  Inc. 


7 


•  Management  Reports:  The  quality  control  reports 
generated  by  TRILAB  were  more  complete  than  those 
generated  by  the  AFCLAS  system.  For  example,  the 
TRILAB  system  produced  Levey-Jennings  graphs,  which 
were  unavailable  under  AFCLAS. 

To  summarize,  the  basic  work  flows  in  the  baseline  (AFCLAS)  and 
post-implementation  (TRILAB)  periods  were  similar.  The  major 
differences  were  in  the  results  availability  and  inquiry  capability  of 
TRILAB  compared  with  the  AFCLAS  system:  the  TRILAB  system  notified 
providers  when  STAT  results  were  available;  the  CRTs  were  easier  to 
use  for  inquiry  by  providers;  and  there  were  nine  more  terminals 
available  in  provider  locations.  In  addition,  the  TRILAB  system  was 
more  flexible  and  had  improved  reporting  capabilities. 

F.  DIFFERENCES  IN  TRILAB  SYSTEMS  AT  WRIGHT  PATTERSON  AND  OAKLAND 
NRMC 

The  TRILAB  system  at  Wright  Patterson  is  similar  to  the  system  at 
Oakland  described  in  Volume  II  of  this  report.  The  following  are 
differences  between  the  two  systems: 

•  At  Oakland,  test  requests  were  generated  by  use  of  more 
than  one  form,  and  tests  were  as  yet  not  centrally 
accessioned  at  the  time  of  the  post-implementation  study. 

•  Cumulative  reports  were  not  routinely  produced  at  Wright 
Patterson  except  on  discharge  (but  were  printed  on  request, 
in  the  Medical  Systems  office),  considerably  reducing  the 
volume  of  paper  generated. 

•  At  Wright  Patterson,  the  telephone  number  of  outpatients 
was  included  on  reports,  thereby  facilitating  follow-up 
contact  with  patients. 

•  The  system  at  Wright  Patterson  could  store  150  days  of 
on-line  test  results  at  the  time  of  the  post-implementation 
survey;  at  Oakland,  only  30  days  could  be  stored. 

•  At  Oakland,  the  Coulter  S+  instrument  was  interfaced  to 
the  computer  system;  at  Wright  Patterson,  Coulter  results 
were  entered  manually  due  to  sequencing  problems  with 
the  Coulter  instrument. 
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II.  APPROACH 


A.  SURVEY  DESIGN 

The  baseline  and  post-implementation  questionnaires  were 
developed  based  on  knowledge  of  the  TRILAB  capabilities,  information 
gained  during  visits  to  the  sites,  and  input  from  the  TRIMIS 
Laboratory  Project  Manager  and  TRIMIS  staff.  The  survey  focused  on 
important  areas  of  laboratory  services  and  those  most  likely  to  be 
affected  by  TRILAB.  The  questions  related  to: 

•  efficiency  of  laboratory  operations; 

•  information  storage  and  retrieval  capabilities; 

•  functional  aspects  of  test  request/report  forms; 

•  accuracy  of  laboratory  results. 

Respondents  were  asked  their  level  of  satisfaction  with  current 
performance  of  laboratory  services.  They  were  also  asked  to  rate  the 
importance  of  improvements  in  each  area.  Since  TRILAB  was  expected  to 
have  an  impact  on  availability  of  test  results,  users  (physicians) 
were  also  asked  to  specify  the  service  response  times  (turnaround  time 
from  request  to  receipt  of  laboratory  results)  they  considered 
acceptable  for  different  types  of  tests.  Clinical  laboratory 
personnel  at  Wright  Patterson  were  queried  concerning  their  attitudes 
regarding  some  aspects  of  the  baseline  computer  system,  so  that  the 
AFCLAS  and  TRILAB  systems  could  be  compared.  All  respondents  were 
also  provided  with  an  opportunity  to  give  free-form  comments  regarding 
any  aspect  of  laboratory  services. 

The  post-implementation  questionnaires  differed  only  slightly 
from  the  baseline  questionnaires.  They  included  additional  questions 
regarding  the  perceived  impacts  of  the  TRILAB  system.  Copies  of  the 
post-implementation  survey  instruments  and  summary  results  are  pro¬ 
vided  in  Appendix  C  of  Volume  VI  of  this  report. 
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B.  QUESTIONNAIRE  ADMINISTRATION 

a.  Baseline 

Distribution  and  collection  of  completed  questionnaires  was  the 
responsibility  of  the  site  in  both  surveys.  In  the  fall  of  1981, 
baseline  questionnaires  were  randomly  distributed  to  users 
(physicians,'  physicians'  assistants,  nurses  and  nurse  practitioners, 
corpsmen,  etc.),  laboratory  personnel,  and  patients.  A  total  of  46 
user  questionnaires  completed  by  physicians  was  available  for 
analysis.  The  respondents  included  22  "light"  users  of  laboratory 
services  (0-10  tests/day) ,  20  moderate  users  (10-20  tests/day)  and  4 
"heavy"  users  (21  or  more  tests/day).  The  sample  included  physicians 
from  the  following  departments:  10  from  Internal  Medicine;  8  from 
Gastroenterology;  5  from  Pediatrics;  3  each  from  Orthopedics, 
Psychiatry,  and  Surgery;  2  each  from  Neonatology,  Cardiology,  Aero¬ 
space  Medicine,  and  Ophthalmology;  and  1  each  from  Family  Practice, 
Primary  Care,  Ear,  Nose  and  Throat,  Emergency  Room,  Urology,  and 
Dermatology. 

A  total  of  26  completed  questionnaires  from  clinical  laboratory 
personnel  was  available  for  analysis.  The  respondents  consisted  of  1 
administrator,  3  laboratory  officers,  20  laboratory  technicians,  and  2 
receptionists/ clerks . 

b.  Post-Implementation 

In  the  post-implementation  period  survey  (November  1982) 
questionnaires  were  distributed  in  the  same  manner  to  users,  labora¬ 
tory  personnel,  and  patients.  Responses  available  for  analysis 
included  the  following:  32  physicians  [14  "light"  users  (0-10  tests/ 
day);  11  "moderate"  users  (11-20  tests/day);  and  6  "heavy"  users  (21 
or  more  tests/day)].  The  sample  included  physicians  from  the 
following  departments:  7  from  Internal  Medicine;  4  each  from  Pedia¬ 
trics  and  Surgery;  2  each  from  Mental  Health  and  0B-GYN;  and  1  each 
from  Medical  Oncology,  Psychiatry,  Urology  and  Ear,  Nose  and  Throat. 
Six  were  unspecified.  Questionnaires  were  also  obtained  from  31 
nurses  and  physicians’  assistants,  and  15  administrative  personnel. 
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A  total  of  12  responses  were  received  from  the  clinical 
laboratory  personnel;  8  were  laboratory  technicians,  3  laboratory 
officers,  and  1  was  unspecified. 

Additionally,  questionnaires  were  received  from  35  outpatients. 

C.  QUESTIONNAIRE  ANALYSIS 

Questionnaire  results  were  keypunched  and  analyzed  by  use  of  a 
computer.  Total  responses  to  each  question  are  presented  on  the 
questionnaires  in  Appendix  C  in  Volume  VI  of  this  report  and  of  the 
baseline  report.  For  the  portion  of  the  questionnaire  involving 
satisfaction  and  importance  ratings  or  frequency  of  events  relating  to 
results  availability,  responses  were  weighted,  summed  and  a  weighted 
mean  response  computed.  The  following  weighting  scheme  was  used: 


Weight 

Satisfaction 

Importance 

Frequency 

5 

Very  Satisfied 

Very  Important 

Often 

4 

Somewhat  Satisfied 

Somewhat  Important 

Occasionally 

3 

Undecided 

Undecided 

Undecided 

2 

Somewhat  Dissatisfied 

Somewhat  Unimportant 

Rarely 

1 

Not  at  All  Satisfied 

Not  at  All  Important 

Never 
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III.  RESULTS 


A.  USER  ATTITUDES 
I.  Test  Report  Form 

a.  Baseline 

Physicians  in  the  baseline  period  indicated  that  the  single  most 
important  aspect  of  the  test  report  form  was  the  completeness  of 
information  provided  (weighted  mean  response  of  4.8).  Slightly  less 
important  was  the  conciseness  of  the  report  form  (weighted  mean  of 
4.5). 

On  average,  users  appear  to  be  at  least  somewhat  satisfied  with 
all  aspects  of  the  forms  in  use  during  the  baseline  study.  As  noted 
in  Table  2,  they  generally  expressed  some  satisfaction  with  informa¬ 
tion  completeness  of  the  report  form  (weighted  mean  of  4.2).  The 
levels  of  satisfaction  with  report  conciseness  and  clarity  were 
slightly  lower  (weighted  mean  "esponse  of  3.7  and  3.9).  Users  were 
less  satisfied  with  the  indication  of  normal /abnormal  values  (3.6). 

b.  Post-Implementation 

As  shown  in  Table  2,  physicians,  nurses,  and  administrators  were, 
for  the  most  part,  satisfied  with  the  test  report  form  during  the 
post-implementation  study.  User  respondents  were  most  satisfied  with 
the  clarity  of  the  printout  and  the  identification  of  abnormal  values 
(weighted  means  of  4.7,  or  "very  satisfied").  These  and  other 
responses  regarding  laboratory  report  forms  are  shown  in  Table  2. 

c.  Baseline  and  Post-Implementation  Comparison 

As  can  be  seen  from  Table  2,  physicians  who  responded  to  the 
post-implementation  survey  felt  more  positively  about  the  test  report 
form  used  with  TRILAB  than  with  that  used  with  AFCLAS;  all  aspects 
measured  showed  greater  satisfaction.  The  most  marked  improvement  was 
in  the  indication  of  abnormal  values,  which  had  a  weighted  mean  of  3.6 
in  the  baseline  and  4.7  in  the  post-implementation  periods.  The 
clarity  and  conciseness  of  information  on  the  report  form  also  showed 
increases  in  satisfaction,  of  0.8  and  0.6  in  average  scale  ratings. 
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Weighted  mean  response  was  obtained  by  assigning  values  of  1  through  5  to  the  categories  very  satisfied,  somewhat  satisfied, 
undecided,  somewhat  unsatisfied,  and  very  unsatisfied,  and  dividing  the  sum  by  the  number  of  responses. 


3.  Clinical  Laboratory  Performance 

a.  Baseline 

When  asked  to  rate  the  importance  of  performance  aspects  of  the 
current  laboratory  services  in  the  baseline  survey,  physicians  were 
virtually  unanimous  in  considering  the  accuracy  of  results  as  "very 
important." 

In  the  baseline  survey,  attitudes  regarding  satisfaction  with 
accuracy  of  test  results  were  fairly  evenly  divided,  as  reflected  in 
the  weighted  mean  of  3.0.  Overall,  users  appeared  to  be  slightly 
dissatisfied  with  service  response  time  for  ER  and  inpatient  STAT,  and 
inpatient  routine  results  (weighted  means  of  2.8,  2.7,  and  2.8, 
respectively)  and  undecided  (weighted  mean  of  3.2)  about  response 
times  for  outpatient  tests.  In  each  case,  however,  15  or  more  of  the 
44  respondents  indicated  that  they  were  "somewhat"  or  "very 
satisfied."  As  shown  in  Table  3,  overall  users  appeared  to  be 
"somewhat  satisfied"  with  the  performance  of  the  clinical  laboratory. 

Of  those  users  who  described  their  use  as  heavy  (21  or  more 
tests/day)  only  one  was  "very  satisfied"  with  the  overall  operations 
of  the  laboratory;  the  others  were  not  very  satisfied.  For  the  most 
part  these  users  were  most  dissatisfied  with  turnaround  times  for 
laboratory  tests.  Additionally,  they  were  somewhat  dissatisfied  with 
their  ability  to  get  information  such  as  laboratory  results  and 
summaries  of  patient  data  bases.  Light  and  moderate  users  were 
consistently  more  satisfied  than  heavy  users  with  these  aspects  of 
services.  All  heavy  users  emphasized  the  importance  of  laboratory 
functions  to  their  work. 

b.  Post-Implementation 

As  can  be  seen  from  Table  3,  generally  users  were  slightly  less 
than  "somewhat  satisfied"  with  most  measures  of  performance  of  the 
clinical  laboratory  in  the  post-implementation  period.  They  were, 
however,  between  "somewhat"  and  "very  satisfied"  with  the  accuracy  of 
results  that  the  laboratory  produces  (nurses  4.3;  physicians  4.1; 
administrators  4.2).  Heavy  users  among  physicians  were  "very 
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satisfied"  with  the  accuracy  of  results  (weighted  mean  4.7);  a 
slightly  lower  level  of  satisfaction  was  exhibited  by  light  and 
moderate  users  (4.0).  (Appendix  C  in  Volume  6  presents  weighted  mean 
satisfaction  levels  for  light,  heavy,  and  moderate  users.) 

Satisfaction  with  turnaround  times  for  laboratory  results  showed 
somewhat  divided  response,  and  tended  to  fall  between  the  categories 
"somewhat  satisfied"  and  "undecided."  As  can  be  seen  from  Table  3, 
all  users  were  least  satisfied  with  the  laboratory's  ability  to  turn 
around  STAT  laboratory  requests  for  inpatient  and  emergency  room 
patients  (overall  weighted  means  of  3.3  for  both).  Nurses  expressed  a 
weighted  mean  satisfaction  of  3.0. 

c.  Baseline  and  Post-Implementation  Comparisons 

The  survey  results  show  that  TRILAB  has  effected  positive  changes 
in  physician  attitudes  towards  laboratory  services,  as  evidenced  in 
Table  3.  Satisfaction  with  all  aspects  of  performance  of  laboratory 
services  increased  from  the  baseline  to  the  post-implementation 
survey.  Physician  opinion  in  the  baseline  was  largely  divided,  as 
indicated  by  weighted  means  of  approximately  3.0.  The  post¬ 
implementation  period  survey  results  indicate  that,  for  the  most  part, 
physicians  were  basically  "satisfied"  with  laboratory  services.  The 
most  marked  change  was  in  satisfaction  with  accuracy  of  laboratory 
results;  respective  weighted  mean  changes  from  the  baseline  to  post¬ 
implementation  were  3.0  to  4.1,  or  "undecided"  to  "somewhat 
satisfied."  There  was  also  a  significant  increase  in  satisfaction 
with  routine  results  turnaround  time,  from  2.8  to  3.7. 

3.  Information  Storage  and  Retrieval 

a.  Baseline 

Baseline  satisfaction  of  physicians  with  the  ability  to  access 
laboratory  results  was  3.3,  or  between  "ut.C  .cided"  and  "somewhat 
satisfied."  On  average,  physicians  appeared  to  be  slightly  dis¬ 
satisfied  with  the  ease  and  timeliness  of  filing  and  the  ability  to 
obtain  status  reports  on  laboratory  tests,  and  only  slightly  satisfied 
with  three  other  aspects.  The  availability  of  cumulative  laboratory 
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summaries  on  patients  rated  the  highest  overall  satisfaction  (weighted 
mean  response  of  3.7,  indicating  respondents  were  "somewhat 
satisfied").  Other  levels  of  satisfaction  with  information  storage 
and  retrieval  are  presented  in  Table  4. 

b.  Post-Implementation 

On  average,  uses  were  "somewhat  satisfied"  with  test  result 
information  storage  and  retrieval  capabilities  in  the  post¬ 
implementation  period.  As  can  be  seen  from  Table  4,  weighted  mean 
responses  fell  between  3.3  and  4.4.  All  users  were  equally  satisfied 
with  TRILAB's  ability  to  access  test  results  (weighted  mean  of  4.4,  or 
between  "very  satisfied"  and  "somewhat  satisfied"). 

Of  the  user  groups,  nurses  were  least  satisfied  with  TRILAB 
system  ability  to  provide  hard-copy  results  for  STAT  results  (weighted 
mean  response  2.8).  The  overall  weighted  mean  for  user  satisfaction 
with  this  aspect  of  TRILAB  information  storage  capabilities  was  3.3, 
with  21  of  the  77  respondents  expressing  some  dissatisfaction. 

As  can  be  seen  from  Table  4,  users  were  quite  satisfied  with  the 
ability  to  search  the  patient  data  base,  access  laboratory  results, 
and  the  timeliness  of  results  filing.  Weighted  means  of  approximately 
4,4  indicate  that  providers  were  between  "very  satisfied"  and 
"somewhat  satisfied." 

c.  Baseline  and  Post-Implementation  Comparison 

Physicians  in  the  post-implementation  period  were  more  satisfied 
with  information  storage  and  retrieval  capabilities  than  in  the 
baseline.  Overall  physicians  were  "somewhat  satisfied"  with  all 
aspects  of  information  storage  and  retrieval.  The  most  narked  change 
in  satisfaction  appears  to  have  been  with  the  aspects  of  information 
storage  and  retrieval  and  the  ability  to  access  laboratory  results, 
probably  due  to  the  greater  flexibility  and  ease  of  data  retrieval  of 
TRILAB  compared  to  the  AFCLAS  system.  Both  of  these  aspects  showed  an 
improved  satisfaction  with  a  difference  of  1.0  and  1.1  (or  from 
basically  "undecided"  to  "somewhat  satisfied").  Other  changes  in 
satisfaction  from  the  baseline  to  post-implementation  are  shown  in 
Table  4. 
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BASELINE  AND  POST-IMPLEMENTATION  ATTITUDES  OF  USERS  OF  CLINICAL  LABORATORY  SERVICES 
REGARDING  LABORATORY  INFORMATION  STORAGE  AND  RETRIEVAL  CAPABILITIES  AT  WRIGHT  PATTERSON  MEDICAL  CENTER  AND  REGIONAL  HOSPITAL3 
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4.  Acceptable  Turnaround  Times  for  Clinical  Laboratory  Services 

a.  Baseline 

The  baseline  distribution  of  responses  regarding  acceptable 
turnaround  times  (time  from  request  to  receipt  of  result)  is  shown  in 
Figure  3.  As  can  be  seen,  virtually  no  users  considered  a  turnaround 
time  in  excess  of  60  minutes  to  be  acceptable  for  inpatient  or 
emergency  STAT  tests.  Most  respondents  considered  a  turnaround  time 
of  24  hours  or  less  acceptable  for  most  routine  tests,  though  18 
respondents  considered  times  of  37-48  hours  acceptable  for  routine 
tests  for  outpatients. 

b.  Post-Implementation 

Figure  4  illustrates  acceptable  turnaround  times  for  users  in  the 
post-implementation  period.  As  in  the  baseline  period,  few  users 
found  STAT  turnaround  times  greater  than  60  minutes  to  be  acceptable. 
Nurses,  in  particular,  expected  short  turnaround  times  (30  minutes  or 
less)  for  STAT  tests.  Most  users  felt  turnaround  time  for  routine 
tests  of  up  to  24  hours  was  acceptable.  Other  measures  of  acceptable 
turnaround  times  are  shown  in  Figure  4. 

Mean  acceptable  turnaround  times  reported  by  physicians  for 
laboratory  tests  in  the  post  implementation  were  as  follows: 

Inpatient  STAT:  44  minutes 

Inpatient  Routine:  13  hours 

Emergency  STAT:  32  minutes 

Outpatient  Routine:  26  hours 

c.  Baseline  and  Post-Implementation  Comparison 

User  expectations  for  turnaround  times  for  laboratory  tests 
remained  consistent.  For  example,  in  both  periods  few  users  found  a 
STAT  turnaround  time  of  greater  than  60  minutes  acceptable.  A 
comparison  of  Figures  3  and  4  shows  very  slight  changes  from  the 
baseline  to  the  post-implementation  survey. 

5.  Frequency  of  Events  Relating  to  the  Availability  of  Test  Results 

a.  Baseline 

As  shown  in  Table  5,  the  most  frequent  action  required  of  a 
physician  to  access  laboratory  results  in  the  baseline  survey  was 
telephone  calls  to  the  laboratory  (a  weighted  mean  response  of  4.4, 
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d  mean  was  obtained  by  assigning,  values  ot  l  through  5  through  the  categories  "often",  "occasionally",  "undoci 
and  "never",  respectively,  and  dividing  the  result  by  the  number  of  responses. 


indicating  telephone  calling  happened  "fairly  often") .  Other  events 
relating  to  obtaining  laboratory  test  results  were  largely  perceived 
as  happening  "occasionally." 

b.  Post-Implementation 

As  shown  in  Table  5,  weighted  mean  physician  responses  concerning 
the  frequency  of  certain  events  with  TRILAB  fell  between  3.0  and  3.4, 
or  between  "occasionally"  and  "undecided."  It  appears  that  users 
overall  felt  the  least  frequently  occurring  event  with  TRILAB  was 
unnecessary  duplication  of  information  (overall  weighted  mean  2.8). 
The  frequency  of  telephone  calls  to  the  laboratory  was  the  most 
frequently  occurring  event  of  all  those  presented  on  Table  5. 
Respondents  felt  that  this  occurred  somewhere  between  "occasionally" 
and  "undecided,"  or  a  weighted  mean  of  3.4. 

c.  Baseline  and  Post-Implementation  Comparison 

Table  5  compares  attitudes  of  physicians  surveyed  in  the  baseline 
and  post-implementation  with  regard  to  the  frequency  of  problem  events 
in  getting  laboratory  test  results  (a  negative  shift  in  frequency 
indicates  an  event  happens  less  commonly).  As  can  be  seen  from  the 
table,  there  appears  to  be  significant  decreases  (of  1.3  in  weighted 
response)  in  the  frequency  of  telephone  calls  to  the  laboratory  as  a 
result  of  the  installation  of  TRILAB  and  in  unnecessary  duplication  of 
information.  Decreases  (of  0.6)  were  measured  for  tests  repeated  due 
to  filing  delays  or  to  inaccurate  results.  All  activities  required  to 
follow-up  on  laboratory  test  results  were  reported  to  occur  less 
frequently  with  TRILAB. 

6.  TRILAB  System  Availability  and  Frequency  of  Events 


The  post-implementation  survey  included  questions  about  the 
availability,  accuracy  and  reliability  of  the  TRILAB  system.  It 
appears  from  the  survey  that  users  overall  were  at  least  "somewhat 
satisfied"  with  these  aspects. 

Users  were  also  asked  how  often  they  employed  the  inquiry 
capabilities  of  the  TRILAB  system  to  obtain  test  results.  Users 
reported  that  most  often  they  relied  on  TRILAB  to  obtain  tests  results 
from  the  previous  day  (weighted  mean  of  4.4,  or  between  "often"  and 
"occasionally").  Frequency  of  obtaining  results  of  tests  over  one 
week  ago  was  between  "rare"  and  "occasionally." 
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In  the  post-implementation  survey,  23  respondents  who  had  worked 
at  the  facility  before  the  installation  of  TRILAB  were  asked  to  rate 
the  relative  frequency  of  common  laboratory  events  such  as  telephone 
calls  to  the  laboratory,  and  tests  repeated  due  to  lost  results, 
filing  delays,  or  inaccurate  results,  since  TRILAB.  Most  respondents 
felt  that  most  common  laboratory  problems  occurred  less  frequently 
with  TRILAB  than  with  the  previous  system.  Physician  responses  fell 
between  "undecided"  and  "less  frequently"  with  regard  to  the  relative 
frequency  of  delay  in  filing  and  necessity  of  tests  being  repeated  due 
to  lost  results.  As  can  be  seen  from  the  table,  physicians,  nurses, 
and  administrators  felt  similarly  about  the  relative  frequency  of 
events . 

B.  ATTITUDES  OF  CLINICAL  LABORATORY  PERSONNEL 

This  section  summarizes  the  results  of  the  baseline  and 
post-implementation  surveys  of  clinical  laboratory  personnel.  As 
noted  in  Chapter  II,  only  12  responses  were  obtained  in  the 
post-implementation  survey,  so  that  differences  in  measured  attitude 
scale  ratings  between  baseline  and  post-implementation  periods  involve 
a  degree  of  uncertainty  and  small  differences  should  be  interpreted 
with  caution. 

1.  Efficiency  of  Laboratory  Operations 

a.  Baseline 

In  order  to  ascertain  which  of  the  common  daily  laboratory 
activities  were  most  important  to  the  clinical  laboratory  personnel 
surveyed,  they  were  asked  to  rate  the  importance  of  such  events,  and 
estimate  time  spent  in  activities  such  as  acquiring  test  results, 
logging  results,  etc.  The  activity  that  personnel  found  most 
important  was  the  time  they  spent  between  examination  of  specimen/ 
culture  and  the  time  it  took  to  complete  the  report  of  test  results. 
The  survey  revealed  a  weighted  mean  importance  level  of  4.8,  or  a 
nearly  unanimous  response  of  "very  important." 

Second  most  important  to  clinical  laboratory  personnel  in  the 
category  of  laboratory  time/efficiency  was  medical  personnel 
familiarity  with  laboratory  operations.  The  survey  revealed  a  mean 
importance  level  of  4.6;  over  two-thirds  of  the  respondents  (16)  felt 
that  the  user's  familiarity  with  the  laboratory  was  "very  important". 
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Laboratory  personnel  in  the  baseline  felt  that  the  most  important 
aspect  of  laboratory  performance  was  the  accuracy  of  the  results  that 
the  laboratory  produces.  Personnel  were  unanimous  in  considering  this 
"very  important"  (a  weighted  mean  response  of  5.0).  The  baseline 
survey  showed  that  laboratory  personnel  considered  quality  control 
"very  important"  (weighted  mean  response  4.9),  or  nearly  as  important 
as  the  accuracy  of  results  overall. 

Satisfaction  ratings  in  the  baseline  period  varied  from  3.2  to 
4.0  for  time  spent  on  preparation  of  department  logs,  reporting  on 
quality  control  information,  training  and  identifying/indicating 
normal  values  (Table  6)  .  A  lower  rating  was  given  to  medical  staff 
familiarity  with  laboratory  operations  (2.3),  indicating  that 
laboratory  staff  were  somewhat  dissatisfied  with  this  aspect  of 
operations . 

Overall  laboratory  performance  received  an  average  scale  rating 
of  3.6. 

Accuracy  of  results  and  quality  control  received  ratings  of  4.3, 
indicating  that  laboratory  staff  were  basically  satisfied  with  these 
aspects  of  laboratory  operations. 

b.  Post-Implementation 

In  the  post-implementation  period,  laboratorians  appeared  to  have 
similar  opinions  as  to  their  satisfaction  with  laboratory  efficiency. 
This  was  evidenced  by  weighted  mean  responses  between  2.8  and  4.0  (see 
Table  6)  .  Satisfaction  level  with  medical  staff  familiarity  with 
laboratory  operations  was  between  "somewhat  unsatisfied"  and 
"undecided"  (2.8  weighted  mean),  indicating  that  laboratory  staff  were 
still  unsatisfied  with  this  aspect  of  laboratory  operations. 
Laboratory  staff  personnel  were  "somewhat  satisfied"  with  time  spent 
identifying  abnormal  values  (weighted  mean  4.0);  this  is  the  aspect  of 
laboratory  efficiency  with  which  they  were  most  satisfied. 

Overall,  laboratory  performance  received  an  average  scale  rating 
of  4.3,  indicating  basic  satisfaction. 
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Laboratorians  were  "somewhat  satisfied"  with  accuracy  of  results 
they  process  in  the  laboratory  (weighted  mean  3.9).  Laboratorians 
were  less  satisfied  with  the  quality  control  of  laboratory  data;  their 
opinion  was  basically  divided  between  "somewhat  satisfied"  and 
"undecided"  (weighted  mean  3.3). 

c.  Baseline  and  Post-Implementation  Comparison 

Table  6  presents  changes  in  laboratory  personnel  attitudes 
regarding  the  efficiency  of  laboratory  operations  and  the  results  that 
the  laboratory  produces.  As  can  be  seen  from  the  table,  personnel 
felt  there  was  little  change  in  laboratory  operations  since  the 
installation  of  TRILAB.  The  noticeable  improvements  in  satisfaction 
since  the  installation  was  in  medical  staff  familiarity  with 
laboratory  operations;  however,  overall  response  is  still  equivocal 
(weighted  mean  2.8).  Personnel  were  slightly  less  satisfied  with  the 
time  they  spend  logging  and  reporting  quality  control  information  and 
in  the  training  of  laboratory  personnel  in  the  post-implementation 
than  they  were  in  the  baseline  period. 

It  appears  that  personnel  were  less  satisfied  with  the  quality 
control  of  laboratory  data  in  the  post-implementation  period  than  they 
were  in  the  baseline  (respective  weighted  means  4.3  and  3.3);  they 
also  felt  somewhat  less  satisfied  with  the  accuracy  of  results  that 
the  laboratory  produces.  These  results  appear  to  be  inconsistent  with 
the  interview  results  (Chapter  IV) ,  which  indicate  that  staff  are  very 
pleased  with  the  improved  quality  control  reporting  capabilities  of 
the  TRILAB  system. 

Overall  satisfaction  with  laboratory  performance,  however, 
increased  from  3.6  to  4.3  from  the  baseline  to  the  post-implementation 
periods. 

2.  Information  Storage  and  Retrieval 

a.  Baseline 

Laboratory  personnel  were  asked  about  their  perceptions  of  the 
various  aspects  of  information  storage  and  retrieval.  The  mean  level 
of  satisfaction  with  results  retrieval  indicated  that  laboratory 
personnel  in  the  baseline  were  basically  satisfied  with  this 
function — 3.7  being  the  mean  satisfaction  response  (Table  7). 


k 

/ti  Arthur  D.  Little,  Inc. 

ft  v  v  v*  '  V  V  .■  «.*  •.  ■  • 


27 


Laboratory  personnel  indicated  their  greatest  dissatisfaction 
with  completeness  of  patient  identification  provided  by  forms,  and  the 
time  spent  tracking  down  this  information  when  it  is  incomplete.  The 
mean  satisfaction  response  obtained  was  2.2,  or  "unsatisfied,"  as 
shown  in  Table  7.  Ten  respondents  indicated  that  they  were  "not  at 
all  satisfied;"  in  other  words,  they  felt  too  much  time  was  spent 
tracking  down  this  information. 

b.  Post-Implementation 

In  the  post-implementation  period,  laboratory  personnel  indicated 
some  dissatisfaction  with  the  time  spent  verifying  patient  information 
(weighted  mean  2.9)  and  with  the  ease  with  which  results  were 
retrieved  (3.3);  see  Table  7. 

c.  Baseline  and  Post-Implementation  Comparison 

Laboratory  personnel  were  divided  with  regard  to  changes  in 
satisfaction  with  information  storage  and  retrieval  capabilities.  In 
the  baseline,  they  were  somewhat  more  satisfied  with  the  speed/ease 
with  which  results  could  be  retrieved,  whereas  in  the 
post-implementation  period  they  were  significantly  more  satisfied  with 
time  spent  verifying  patient  identification  (baseline  2.2;  post¬ 
implementation  2.9).  Table  7  summarizes  these  findings. 

3.  Attributes  of  Computer  Hardware 

a.  Baseline 

The  laboratory  personnel  at  Wright  Patterson  who  responded  to  the 
baseline  survey  felt  that  the  most  important  aspect  of  the  AFCLAS 
computer  system  was  the  efficiency  of  command  levels.  Twenty 
respondents  (weighted  mean  of  4.7)  felt  command  functions  were  "very 
important."  Personnel  appeared  to  be  "somewhat  satisfied"  with  this 
function — a  weighted  mean  response  of  3.9  was  obtained  (Table  7). 
They  were  also  satisfied  with  legibility  of  the  CRT  screen  (4.3). 

b.  Post-Implementation 

As  shown  in  Table  7,  laboratorians  appeared  to  be  "somewhat 
satisfied"  with  the  attributes  of  the  TRILAB  computer  hardware  and 
command  functions.  The  weighted  mean  of  4.4  indicates  that  they  were 
between  "very  satisfied"  and  "somewhat  satisfied"  with  the  legibility 
of  the  CRT  screen. 
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c.  Baseline  and  Post-Implementation  Comparison 

As  can  be  seen  from  Table  7,  attitudes  regarding  satisfaction 
with  computer  hardware  remained  basically  the  same  from  the  baseline 
to  the  post-implementation  period.  Personnel  were  "somewhat 
satisfied"  with  the  legibility  of  the  CRT  screen  in  both  periods 
(weighted  means  of  4.3  and  4.4). 

4.  Frequency  of  Laboratory  Problems 

a.  Baseline 

In  order  to  ascertain  the  perception  of  frequency  of  day-to-day 
occurrences  that  interrupt  laboratory  routine,  respondents  were  asked 
to  indicate  the  frequency  with  which  these  activities  occur,  from 
"never"  to  "often."  Nearly  all  respondents  in  the  baseline  felt  that 
most  frequently  occurring  activity  was  telephone  calling  to  the 
laboratory.  A  weighted  mean  response  of  4.8  indicates  that  this 
happened  "often,"  as  shown  in  Table  8.  Related  to  this  was  the  time 
personnel  spend  telephoning  STAT  results  to  the  ward,  and  nearly  as 
many  respondents  felt  that  this,  too,  was  a  frequent  occurrence.  A 
mean  response  of  4.4  was  obtained.  Other  perceptions  of  frequency  of 
events  relating  to  test  results  availability  are  noted  in  Table  8. 

b.  Post-Implementation 

As  can  be  seen  from  Table  8,  respondents  in  the  post¬ 
implementation  period  had  divided  opinions  as  to  their  perceptions  of 
the  frequency  of  laboratory  events.  It  appeared,  however,  that  the 
least  frequently  occurring  event  since  TRILAB's  installation  had  been 
the  telephone  calling  of  STAT  results  to  units  (weighted  mean  2.4,  or 
for  the  most  part,  "rarely."  Weighted  means  indicated  that  labora- 
torians  still  spent  time  on  manual  record  keeping  (3.7,  or 
"occasionally") . 

c.  Baseline  and  Post-Implementation  Comparison 

Laboratory  personnel  were,  overall,  in  agreement  that  frequency 
of  common  laboratory  discrepancies  decreased  in  the  post¬ 
implementation  period.  In  particular,  there  was  a  definite  feeling 
that  telephone  calls  to  the  laboratory  had  decreased  significantly 
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BASELINE  AND  POST-IMPLEMENTATION  ATTITUDES  OF  CLINICAL  IJVBORATORY  PERSONNEL  RECARDINC 
FREQUENCY  OF  EVENTS  RELATING  TO  LABORATORY  RESULTS  AND  RELATED  ACTIVITIES  AT 
WRICIIT  PATTERSON  MEDICAL  CENTER  AND  REGIONAL  HOSPITAL3 
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A  weighted  mean  was  obtained  by  assigning  values  of  1  through  5  to  the  categories  "often",  "occasionally",  "undecided 
'rarely",  and  "never"  and  dividing  the  sum  by  the  number  of  responses. 
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(baseline  weighted  mean  4.8,  post-implementation  3.5).  Time  spent  to 
telephone  STAT  results  to  units  also  was  perceived  to  decrease  sub¬ 
stantially  (from  4.4  to  2.4).  Personnel  also  appeared  to  feel  that 
there  was  a  significant  decrease  in  transcription  and  calculation 
errors  (4.3  and  3.3,  baseline  and  post-implementation,  respectively), 
possibly  because  of  improved  highlighting  of  abnormal  values.  Table  8 
presents  other  comparative  values  regarding  frequency  of  laboratory 
events. 

5.  Expected  and  Realized  Improvements 

a.  Baseline 

When  personnel  were  asked  in  the  baseline  survey  what 
improvements  in  the  new  system  would  be  most  important  to  them,  most 
indicated  that  improved  accuracy  of  test  results  was  their  first 
priority,  followed  by  improved  efficiency,  change  in  format  of 
laboratory  request/reports,  and  finally  speed/ease  of  information 
retrieval.  Laboratory  staff  were  not  asked  to  indicate  the  importance 
of  these  improvements  on  a  weighted  scale. 

b.  Post-Implementation 

It  appears  that  accuracy  of  results,  the  area  clinical  laboratory 
personnel  felt  was  in  greatest  need  of  improvement,  was,  in  fact,  the 
area  that  improved  the  most  (weighted  mean  3.7).  This  result  is 
inconsistent,  however,  with  the  results  presented  in  Section  C.l, 
which  showed  a  small  decrease  in  satisfaction  with  results  accuracy. 
As  mentioned  above,  the  inconsistency  may  be  due  to  the  small  number 
of  responses  received  (12). 

Other  perceptions  of  improvements  resulting  from  the  installation 
of  TRILAB  are  shown  in  Table  9,  including  improvements  in  ease  of 
information  storage  and  retrieval,  number  of  telephone  calls  to  the 
laboratory,  and  efficiency  of  laboratory  operations. 

6.  Relative  Frequency  of  Discrepancies  with  TRILAB 

As  can  be  seen  from  Table  9,  it  appears  that  personnel  feel  that 
common  laboratory  discrepancies  happened  with  slightly  less  relative 
frequency  than  before  TRILAB  was  installed.  In  particular,  personnel 
felt  that  they  "rarely"  spent  time  calling  results  to  units  (weighted 
mean  2.1).  Other  results  are  presented  in  Table  9. 
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Weighted  mean  frequency  and  improvement  were  obtained  by  assigning  values  of  1  through  5  to  the  categories  "often" 
through  "never"  and  "very  important"  through  "not  at  all  important",  respectively,  and  divided  the  sums  by  the 
number  of  responses. 


C.  PATIENTS 


No  patient  data  were  available  for  analysis  in  the  baseline 
period;  therefore,  only  post-implementation  data  will  be  discussed 
here.  In  the  post-implementation  period,  35  outpatients  responded  to 
the  survey.  Table  10  presents  patient  responses  to  the  post¬ 
implementation  survey.  It  appears  that  these  patients  were  on  average 
"somewhat  satisfied"  with  clinical  laboratory  services  at  Wright 
Patterson,  including  time  waiting  to  be  served  in  the  laboratory. 
Delayed  or  repeated  tests  were  considered  to  occur  on  average  between 
"rarely"  and  "never." 
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ATTITUDES  OF  OUT -PATIENTS  RECARDING  SATISFACTION  WITH  CLINICAL  LABORATORY  SERVICES  AND 
FREQUENCY  OF  EVENTS  CONCERNING  THE  CLINICAL  LABORATORY  AT 
WRTGHT  PATTERSON  MEDICAL  CENTER  AND  RECIONAL  HOSPITAL3 


A  weighted  mean  was  obtained  by  assigning  values  of  5  through  1  to  the  categories  of  "often",  "occasionally",  "undecided" 
rarely",  "never",  and  dividing  this  sum  by  the  number  of  responses. 


IV.  INTERVIEW  RESULTS 


A.  INTRODUCTION 

In  December  of  1982,  approximately  20  physicians,  nurses, 
physicians'  assistants,  ward  clerks  and  laboratory  personnel  were 
interviewed  in  order  to  gather  additional  information  on 
post-implementation  findings  at  Wright  Patterson.  The  following 
discusses  both  the  benefits  and  the  problems  of  the  TRILAB  system  as 
perceived  by  the  providers  and  laboratorians  interviewed. 

B.  USERS 

1.  Benefits 

•  Decrease  in  Telephone  Calls  to  Laboratory:  All  personnel 
interviewed  agreed  that  the  amount  of  time  they  spent 
telephoning  the  laboratory  to  retrieve  results  had 
decreased  substantially  since  the  installation  of  TRILAB, 
particularly  with  regard  to  STAT  results.  Most  personnel 
indicated  that  they  rarely  called  the  laboratory  since  the 
implementation  of  TRILAB. 

•  Visits  to  the  Laboratory:  Users  interviewed  reported 
that  they  rarely,  if  ever,  had  to  visit  the  laboratory 
to  retrieve  results  since  the  installation  of  TRILAB. 

Nurses  and  ward  clerks  were  most  concerned  with  this 
activity,  as  generally  it  had  been  their  responsibility 
to  retrieve  results.  With  TRILAB,  the  only  reason 
nurses/ward  clerks  visited  the  laboratory  was  to  bring 
down  STAT  specimens. 

•  Filing  of  Results:  Instead  of  having  to  sift  through 
charts,  users  now  relied  on  terminals  to  find  patient 
laboratory  data.  Interviewees  also  indicated  that  the 
cumulative  reports  made  available  by  TRILAB  saved  time 
in  searching  for  retrospective  data. 

According  to  interviews  in  several  wards  and  clinics,  nursing  staff 
estimated  that  an  average  of  3.5  hours  per  day  at  each  nursing  station 
with  a  terminal  had  been  saved  due  to  the  above  three  effects,  with 
the  major  savings  coming  from  reduced  telephone  calls.  This  average 
took  into  account  time  spent  on  all  three  shifts. 
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•  Errors  and  Duplications  of  Tests:  The  impact  of  TRILAB  on 
errors  involved  in  laboratory  test  procedures  appeared  not 
to  have  been  as  great  as  in  other  areas  such  as  telephone 
call  reductions  and  management  reporting  capabilities. 

Some  users  indicated  that  there  was  less  duplication  of 
tests,  perhaps  resulting  from  the  fact  that  results  appeared 
on  the  terminal  as  they  became  available,  and  there  was  less 
chance  that  they  would  be  lost. 

•  Retrieval  of  Information:  Users  interviewed  relied  heavily 
on  TRILAB's  information  storage  and  retrieval  capabilities. 

All  comments  in  this  regard  were  highly  positive.  This 
capability  was  reported  to  provide  a  great  deal  of  information 
to  users. 

2.  Problems 

Most  users  interviewed  felt  that  there  were  no  major  problems 
with  the  TRILAB  system.  Comments  included  the  lack  of  quickly 
available  hard-copy  STAT  results,  and  the  need  for  more  terminals  on 
wards/clinics. 

C.  CLINICAL  LABORATORY  PERSONNEL 
1.  Benefits 

The  following  were  perceived  by  clinical  laboratory  personnel  to 
be  benefits  of  the  TRILAB  system: 

•  Management  Reporting:  Laboratorians  interviewed  felt  that 
TRILAB  accomplished  management  reporting  tasks  more  effi¬ 
ciently  than  in  the  baseline.  They  appeared  to  use  the 
system  effectively  to  generate  workload  statistics. 

•  Quality  Control  Reporting:  Laboratorians  interviewed  felt 
that  the  quality  control  capabilities  of  TRILAB  were 
excellent.  One  staff  member  in  the  Clinical  Chemistry 
Department  felt  that  TRILAB  Levey-Jennings  quality  control 
charts  were  particularly  helpful,  and  estimated  they  saved 
his  section  about  one  day  of  staff  time  per  month. 

•  Volume  of  Telephone  Calls  to  the  Laboratory:  Laboratorians 
in  all  sections  indicated  a  very  significant  decrease  in 
telephone  calls  to  the  laboratory. 
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•  Errors  and  Duplication  of  Tests:  It  was  felt  by  interviewees 
that  duplication  and  errors  in  tests  occurred  "somewhat 
less"  than  in  the  baseline  period. 

2.  Problems 

Laboratorians  articulated  a  few  problems  that  they  have  had  with 
TRILAB,  the  foremost  of  them  being  the  response  time  of  the  system. 
Some  personnel  felt  that  terminal  response  time,  particularly  when 
inputting  corrections,  was  long.  Most  laboratory  personnel  felt  that 
there  were  minor  issues  related  to  the  newness  of  the  TRILAB  system, 
and  that  these  would  be  worked  out  in  time. 

D.  CONCLUSIONS 

Both  the  survey  data  and  interviews  suggest  that  the  TRILAB 
system  is  perceived  as  a  significant  improvement  over  the  previous 
AFCLAS  system  which  it  replaced.  This  improvement  in  perceived 
capabilities  appears  to  be  related  not  only  to  the  fact  that  the 
system  supports  more  terminals  in  patient  loactions,  but  also  to  the 
improved  capability  of  the  TRILAB  system  for  results  inquiry 
(including  notification  to  provider  locations  of  STAT  result 
availability),  improved  management  and  quality  control  reporting,  and 
greater  flexibility  in  implementing  program  and  report  format  changes. 

Providers  indicated  significant  improvement  between  the  baseline 
and  post-implementation  periods  with  respect  to  the  following: 

•  clarity  of  information  on  test  report  forms; 

•  indication  of  normal/abnormal  values  on  the 
test  reports; 

•  accuracy  of  results; 

•  length  of  time  between  result  requests  and  results, 
particularly  for  inpatient  routine  tests; 

•  improved  ability  to  access  laboratory  results  and 
retrieve  previous  laboratory  test  data; 

•  improved  ease  and  timeliness  of  filing  results; 

•  significant  reduction  in  frequency  of  telephone 
calls  to  the  laboratory; 

•  reduction  in  unnecessary  duplication  of  information. 
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Nursing  staff  estimated  that  the  reduction  in  telephone  calls  to  the 
laboratory,  particularly  with  regard  to  STAT  results,  resulted  in 
average  savings  of  3.5  hours  per  day  at  each  station  with  a  terminal. 
It  was  also  felt  that  there  were  fewer  visits  to  the  laboratory  to 
retrieve  the  results,  a  possible  small  reduction  in  number  of  test 
result  transcription  errors  and  duplication  of  tests,  and  a  general 
improvement  in  the  capability  of  retrieving  results  using  the 
terminals,  instead  of  attempting  to  find  results  in  charts. 

Laboratory  staff  reported  that  in  general  overall  performance  of 
the  laboratory  had  increased  somewhat  between  the  baseline  and 
post-implementation  periods  and  felt  that  time  spent  on  verification 
of  the  patient  information  had  decreased,  as  well  as  indicating 
significant  reductions  in: 

•  telephone  calls  to  the  laboratory; 

•  time  spent  telephoning  STAT  results  to  units; 

•  transcription  errors. 

Although  the  survey  results  suggest  that  respondents  were  less 
satisfied  with  quality  control  aspects  of  TRILAB  compared  with  the 
previous  system,  the  interviews  indicated  that  laboratory  staff  were 
very  pleased  with  the  improved  management  and  quality  control 
reporting  capabilities  of  the  TRILAB  system,  and  specifically  cited 
the  improved  quality  control  reporting  capabilities  of  TRILAB. 

To  summarize,  both  providers  and  laboratory  staff  in  general 
indicated  the  TRILAB  system  was  an  improvement  over  the  AFCLAS  system. 
The  major  perceived  benefits  appear  to  be  associated  with  the  improved 
flexibility  and  information  retrieval  capabilities  of  TRILAB,  which 
has  resulted  in  a  significant  reduction  in  telephone  calls  both  to  and 
from  the  laboratory,  in  providing  users  and  laboratory  staff  improved 
information  retrieval  capabilities  for  test  results,  and  in  laboratory 
management  reporting. 
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